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Abstract 

Seabasing  has  been  identified  as  a  critical  future  joint  military  capability  for  the  United 
States.  The  complexity  of  the  Seabasing  architecture  requires  a  coordinated  development 
effort  to  address  identified  issues  and  to  create  a  joint  Seabasing  system-of-sy stems.  New 
technologies  that  provide  updated  capabilities  are  needed  to  make  the  Seabasing  concept 
feasible.  It  is  essential  to  identify  the  capabilities  required  of  these  new  technologies  and 
to  quantify  the  impact  of  capability  tradeoffs  on  the  Seabasing  concept. 

This  paper  presents  a  quantitative  framework  to  assess  the  impacts  of  new  technologies 
and  systems  on  the  overall  Seabasing  system-of-systems.  An  architecture-driven 
approach  is  employed  to  develop  a  discrete  event  model  of  the  Sea  Base-to-Objective 
system.  Surrogate  models  are  constructed  to  enable  rapid,  probabilistic  design  for 
capability. 

Compared  with  previous  methods,  the  proposed  approach  enables  decision  makers  to 
make  informed  decisions  during  the  requirements  definition  and  conceptual  phases  and 
offers  the  potential  to  reduce  the  time  and  cost  needed  to  develop  a  design  that  meets  or 
exceeds  customer  requirements. 
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1  Background 

As  outlined  by  Admiral  Vem  Clark,  Sea  Power  21  is  a  strategic  vision  enacted  by  the 
United  States  Navy  to  navigate  21st  century  challenges  by  aligning,  organizing, 
integrating  and  transforming  its  efforts  to  reach  maximum  combat  power.  In 
accomplishing  these  objectives,  Sea  Power  21  outlines  capabilities  that  must  be  employed 
to  counter  the  dangers  of  varied  and  deadly  threats  that  pose  challenges  to  national 
security  and  future  war  fighting.  The  capabilities  that  ultimately  define  Sea  Power  21  are 
Sea  Strike,  Sea  Shield  and  Seabasing. 

Sea  Strike  outlines  a  strategy  for  offensive  power;  Sea  Shield  outlines  a  strategy  for 
global  defensive  assurance;  as  Seabasing  outlines  strategies  for  operational  independence 
[1]. 

The  Seabasing  capability  serves  as  a  base  and  supports  Sea  Shield  and  Sea  Strike. 
Seabasing  will  allow  “ pre-positioned  [war  fighting]  capabilities  for  immediate 
employment,  enhanced  joint  support  from  a  fully  netted,  dispersed  naval  force,  strengthen 
international  coalition  building,  increase  joint  force  security  and  operational  agility,  and 
minimize  operational  reliance  on  shore  infrastructure.  ”  This  concept  enables  the  United 
States  to  augment  force  protection  by  reducing  vulnerabilities  [1], 

A  vulnerability  gap,  due  the  differences  in  the  rate  at  which  the  initial  landing  force  can 
be  reinforced  and  the  rate  at  which  the  enemy  force  can  reinforce.  This  allows  the 
possibility  of  the  enemy  projecting  a  greater  force.  Figure  1  clearly  depicts  how  the 
vulnerability  gap  impacts  the  level  of  combat  power  over  time. 


The  Vulnerability  Gap  Results 
from  a  Net  Force  Deficit 


Figure  1:  Vulnerability  gap  resulting  from  net  force  deficit  [2] 


Sea  Base-Enhanced  Projected  Force  amplifies  combat  power  by  sustaining  troops  over 
time  and  removes  the  transitory  impact  of  the  Initial  Forces  seen  in  Figure  1 .  Figure  2 
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depicts  a  reduction  in  the  Vulnerability  Gap  with  Seabasing  and  displays  an  improvement 
in  the  sustained  level  of  combat  power  [2], 


Sea-Based  Sustainment  can  Reduce  or 
Eliminate  Vulnerability  Gap 


Figure  2:  Reduction  of  vulnerability  gap  from  Sea  Based  sustainment  [2] 

To  achieve  the  desired  Seabasing  objectives  and  to  reduce  this  Vulnerability  Gap,  a 
Concept  of  Operations  (CONOPS)  has  been  developed.  Often  referred  to  as  CEASaR, 
the  CONOPS  for  Seabasing  is  broken  into  five  organizational  phases:  Close,  Assemble, 
Employ,  Sustain,  and  Reconstitute.  These  phases  make  the  mobility  of  joint  forces 
continuous  and  constant,  over  time  -  thus  reducing  the  Vulnerability  Gap. 

In  the  Close  phase,  a  Marine  Expeditionary  Brigade  (MEB)  sized  force  is  brought  to  the 
Sea  Base.  This  phase  is  defined  by  its  ability  to  rapidly  move  joint  forces  to  an  area  of 
crisis.  Defined  by  its  ability  to  safely  allow  the  integration  of  the  joint  force,  the 
Assemble  phase  provides  a  secure  environment  in  preparation  of  troop  employment. 
Through  the  support  of  the  sea  base,  the  Employ  phase  provides  flexibility  to  the  joint 
force  by  meeting  changing  requirements  and  mission  objectives.  This  phase  is  defined  by 
its  ability  to  reduce  the  logistical  footprint  ashore  and  the  force  protection  requirements. 
The  Sustain  phase  provides  a  steady  flow  of  support  for  ship  and  shore  based  joint  forces. 
The  Reconstitute  phase  allows  joint  forces  to  rapidly  deploy  joint  forces  again  for 
subsequent  operations  [3], 

This  CONOPS  has  been  further  detailed  with  Measures  of  Performance  (MOPs)  and 
Measures  of  Effectiveness  (MOEs)  meeting  Seabasing  goals.  MOPs  are  “ designed  to 
correspond  to  accomplishment  of  mission  objectives  and  achievement  of  desired  effects  ”. 
MOEs  are  “ designed  to  quantify  the  degree  of  perfection  in  accomplishing  functions  or 
tasks.  ”  The  MOPs  for  meeting  Seabasing  goals  are  as  follows  [3]: 

CLOSE  joint  sea-based  capabilities,  including  elements  of  [Joint  Command  and 
Control],  to  a  [Joint  Operations  Area [  to  support  major  combat 
operations  within  10-14  days  of  execution  order. 
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ASSEMBLE  and  integrate  joint  capabilities  from  the  sea  base  to  support  major 
combat  operations  within  24-72  hours  of  arrival  within  the  [Joint 
Operations  Area]. 

EMPLOY  over-the-horizon  from  the  sea  base  at  least  one  (1)  brigade  for  [Joint 
Forcible  Entry  Operations]  within  a  period  of  darkness  (8-10  hrs). 

SUSTAIN  joint  sea-based  operations,  including  up  to  at  least  two  (2)  joint 
brigades  operating  ashore,  for  an  indefinite  period  using  secure  advanced 
bases  up  to  2000  nm  away;  also  support  selected  joint  maintenance  and 
provide  level  III  medical  within  the  sea  base. 

RECONSTITUTE  one  (1)  brigade  from  ashore  to  the  sea  base  and  reemploy 
within  10-14  days  of  execution  order. 

Figure  3  shows  an  overarching  view  of  Seabasing. 


Seabasing  is  a  viable  and  attractive  concept  but  for  it  to  fully  manifest,  some  points  of 
action  are:  “ exploit  the  advantages  of  sea-based  forces  whenever  possible;  develop 
technologies  to  enhance  on-station  time  and  minimize  maintenance  requirements; 
experiment  with  innovative  employment  concepts  and  platforms;  and,  challenge  every 
assumption  that  results  in  shore  basing  of  Navy  Capabilities”  [1]. 

It  is  not  only  imperative  to  assess  the  overall  Seabasing  Concept,  but  the  assessment  of 
new  technology  and  systems  becomes  just  as  crucial.  The  possibilities  offered  by  new 
technologies  and  systems  require  impact  assessment  for  Seabasing  to  become  a  reality. 
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An  example  is  the  Office  of  Naval  Research’s  development  of  a  new  Sea  Base  Connector 
Transformable-Craft  (T-CRAFT).  Currently  under  Phase  II  of  development,  T-CRAFT 
is  a  prototype  demonstrator  intended  to  be  deployable  from  an  advanced  base  to  the  Sea 
Base  and  used  as  a  surface  connector.  The  operational  aspects  of  T-CRAFT  surpass  any 
current  legacy  platforms,  and  can  be  viewed  in  [4],  To  validate  its  operational  ability  in 
meeting  Seabasing  goals,  an  assessment  addressing  how  these  operational  capabilities 
perform  must  be  conducted. 

2  Literature  Review 

2.1  Assessing  the  Seabasing  Concept 

Seabasing  has  been  identified  as  a  critical  joint  forces  capability  allowing  the  United 
States  to  project  forces  more  rapidly.  The  development  of  such  a  system  requires  a 
coordinated  effort  to  ensure  that  it  meet  the  needs  of  the  joint  forces  [2],  Studies 
conducted  to  date,  have  been  constrained  to  qualitative  assessments.  These  evaluations 
shed  light  on  Seabasing  operations,  but  they  don’t  specifically  identify  system 
performance  in  meeting  Seabasing  MOPs. 

In  2007,  the  Congressional  Budget  Office  conducted  a  study  on  deploying  and  sustaining 
troops  in  Seabasing  operations.  This  study  provided  a  guide  to  system  capabilities  and 
costs,  sustainment  requirements  from  a  resulting  force  and  detail  of  alternatives. 
Although  there  were  well  formed  theories  presented  of  how  these  alternatives  would 
allow  joint  forces  meet  Seabasing  goals,  there  is  no  evidence  proving  that  these  theories 
would  meet  the  Seabasing  MOPs  [5], 

2.2  The  Need  for  Modeling  &  Simulation 

The  use  of  Modeling  and  Simulation  has  become  increasingly  important  in  understanding 
how  complex  systems  behave.  Modeling  and  Simulation  gives  decision  makers  the  ability 
to  identify  manipulate  systems  and  retrieve  system  behavior.  The  Department  of  Defense 
(DoD)  has  strongly  advocated  the  use  of  Modeling  and  Simulation  throughout  the 
acquisition  lifecycle  [6]. 

In  1997,  DoD  formally  adopted  a  vision  for  integrating  modeling  and  simulation  into  the 
acquisition  lifecycle  [7],  Simulated  Based  Acquisition  (SB A)  reduces  time,  resources 
and  risks  associated  with  the  acquisition  process  by  integrating  product  and  process 
development  across  the  acquisition  lifecycle  [8],  This  vision  became  a  mandate  in  2002 
when  DoD  issued  the  directive  5000.01  requiring  the  integration  of  test  and  evaluation  in 
the  process  of  acquisition.  The  integration  of  test  and  evaluation  allows  the  assessment 
of  technical  performance  parameters  and  to  determine  the  operational  impacts  of  acquired 
systems.  Under  this  directive,  mandates  of  modeling  and  simulation  are  formalized  in 
order  to  ‘ facilitate  learning,  assess  technology  maturity  and  interoperability,  facilitate 
integration  into  fielded  forces,  and  confirm  performance  against  documented  capability 
needs  and  adversary  capabilities ”  [9], 
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Recent  studies  have  shown  the  feasibility  and  prowess  of  using  Modeling  and  Simulation 
in  the  acquisition  and  development  of  complex  systems.  In  1996,  the  Director  of  Test, 
Systems  Engineering  and  Evaluation  in  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Logistics  and  Technology  at  the  Department  of  Defense  commissioned  a 
study  to  access  the  use  of  Modeling  and  Simulation  in  the  weapon  systems  acquisition 
process.  This  study  found  that  cost,  schedule,  productivity,  and  quality/performance 
could  be  thoroughly  analyzed  through  Modeling  and  Simulation  and  therefore  aid 
decision  makers  in  the  systems  acquisition  process  [10].  More  recently,  in  2004,  the 
Naval  Post  Graduate  School  used  Modeling  and  Simulation  to  define  capability  gaps  in 
force  projection  while  implementing  Ship-To-Objective  Maneuver  via  the  Seabasing 
Concept.  Through  this  study,  various  architectures  (sensor,  weapon,  and  force)  were 
studied  to  meet  force  projection  goals  in  Seabasing.  The  resulting  findings  include  the 
best  performing  architecture  and  insight  on  how  this  architecture  improves  force 
projection.  With  over  60  students  and  15  faculty  members  working  on  this  project,  the 
man-power  to  produce  such  a  study  is  taxing  [11], 

Modeling  and  Simulation  has  been  regarded  for  its  effectiveness  in  allowing  DoD 
decision  makers  ascertain  the  behavior  of  a  system,  but  its  ability  to  perform  trade-offs  of 
desired  metrics  is  tedious,  time  consuming  and  restrictive.  This  research  intends  to  build 
upon  using  Modeling  and  Simulation  as  a  decision  making  tool  by  developing  a 
quantitative  and  probabilistic  capability  based  trade-off  environment.  The  resulting 
framework  created  will  give  decision  makers  the  ability  to  rapidly  assess  the  impact 
different  technologies  and  systems  have  on  the  Seabasing  concept.  Once  established,  this 
framework  will  allow  validation  of  notional  operational  capabilities  in  new  technologies 
and  systems  and  their  ability  to  meet  Seabasing  MOPs. 

3  Proposed  Approach 

The  proposed  approach  combines  methods  from  Leite  and  Mensh  [12]  and  Kirby  [13], 
Leite  and  Mensh  develop  a  methodology  for  the  generation  of  evaluation  criteria  for 
system  acquisition  modeling  and  simulation  based  on  the  underlying  system 
requirements,  and  stipulate  that  “all  metrics  must  be  traceable  to  requirements  and  all 
requirements  must  be  associated  with  metrics.  ”  Kirby  outlines  a  methodology  for 
technology  identification,  evaluation,  and  selection  in  conceptual  and  preliminary  aircraft 
design.  Kirby  employs  surrogate  modeling  techniques  to  rapidly  evaluate  aircraft 
performance  under  various  technology  combinations  and  to  select  the  optimal  technology 
combination  that  satisfies  performance  and  economics  requirements. 

3.1  STEP  1  Problem  Definition 

The  first  step  consists  of  three  substeps:  identification  of  the  system  of  interest, 
identification  of  the  capabilities  to  be  modeled,  and  identification  of  the  system  metrics  to 
be  studied. 
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Identify  System  of  Interest 

The  first  substep  in  the  development  is  to  identify  the  system  to  be  explored  and  to  scope 
the  problem  to  the  desired  level  of  detail.  This  may  require  decomposition  and/or 
abstraction  of  the  system  to  be  studied. 

Identify  Capabilities  of  Interest 

The  second  substep  is  to  identify  the  system  capabilities  that  will  be  modeled.  This  step 
bounds  the  model.  In  many  cases,  it  is  not  possible  to  model  all  of  the  system’s 
capabilities  due  to  resource  constraints.  Furthermore,  some  systems  may  be  too  complex 
or  undefined  to  be  modeled  in  their  entirety  It  is  assumed  that  the  system  provides  the 
capabilities  that  are  not  modeled  for  all  cases  to  be  studied.  This  is  particularly  true  for 
new  developments  such  as  the  Seabasing  concept. 

Identify  System  Metrics 

The  third  substep  is  to  develop  the  metrics  for  the  capabilities  being  modeled.  These 
metrics  are  defined  for  the  operational  system  and  are  in  the  form  of  measures  of 
performance,  measures  of  effectiveness,  and  measures  of  force  effectiveness. 

3.2  STEP  2  Modeling  and  Simulation 

Identify  Systems  Functionality 

The  first  step  in  the  model  development  is  to  identify  the  system  functions  that  must  be 
modeled  to  test  the  identified  capabilities.  Functions  that  are  not  related  to  the 
capabilities  being  modeled  are  assumed  to  perform  correctly  for  all  cases  being  studied 
and  may  be  represented  by  nominal  inputs. 

Develop  System  Model 

The  next  step  is  to  develop  a  deterministic  model  that  will  quantify  the  impact  of 
capability  tradeoffs  on  the  system.  The  level  of  the  capabilities  and  metrics  determine  the 
scope  of  the  modeling  and  simulation  that  is  required.  Before  the  model  can  be  used  as  a 
representation  of  a  system,  it  must  be  verified  and  validated.  Verification  involves 
tracing  the  model  inputs  through  the  system  functions  and  ensuring  that  the  model 
correctly  implements  the  required  system  functions.  The  model  must  be  validated  against 
a  set  of  inputs  with  known  outputs.  Model  behavior  must  match  that  which  is  expected 
beforehand;  when  performance  anomalies  are  encountered,  developers  must  determine 
whether  an  anomaly  is  due  to  an  incorrect  representation  in  the  model  or  whether  the 
system  itself  is  flawed. 

3.3  STEP  3  Development  of  Tradeoff  Environment 

Development  of  the  tradeoff  environment  entails  sampling  the  model  using  Design  of 
Experiments  (DoE)  in  order  to  develop  a  surrogate  model.  The  surrogate  model  is 
developed  via  regression  of  the  DoE  results.  This  enables  the  development  of  a  rapid 
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tradeoff  environment  that  can  be  explored  in  real-time.  If  the  model  runs  sufficiently 
quickly  or  the  design  space  is  small,  the  model  may  be  executed  directly.  The  surrogate 
model  must  be  validated  against  the  original  model  data  and  in  general  should  not 
extrapolate  to  areas  outside  of  the  design  parameter  ranges. 

3.4  STEP  4  Design  Space  Exploration 

The  final  step  is  to  use  the  tradeoff  environment  to  explore  the  impact  of  capability 
tradeoffs  on  the  system  performance.  A  probabilistic  assessment  of  achieving  required 
system  metrics  is  enabled  by  sampling  the  surrogate  model  using,  Monte  Carlo  or  Latin 
Hypercube  sampling.  Due  to  uncertainty  in  the  design  process,  this  step  is  probabilistic 
rather  than  deterministic  in  nature.  Expressing  the  system  metrics  as  probabilistic 
distributions  has  the  additional  advantage  of  allowing  designers  to  determine  if  a  change 
is  statistically  significant  [14].  Sensitivities  are  studied  to  determine  the  robustness  of 
selected  designs.  Design  space  exploration  is  iterative,  as  the  initial  design  space  may  not 
capture  a  sufficient  number  of  feasible  solutions;  in  this  case,  the  original  model  must  be 
re-evaluated  to  determine  the  cause  of  failure. 

4  Proof  of  Concept 

The  proposed  approach  is  applied  to  study  the  Sea  Base-to-Objective  system  of  the 
Seabasing  concept.  The  goal  is  to  develop  a  quantitative  capability-based  tradeoff 
environment  that  can  be  used  to  explore  the  design  space  in  real-time  and  offer  insight 
into  the  probability  of  meeting  required  metrics. 

4.1  STEP  1  Problem  Definition 

Identify  System  of  Interest 

The  Seabasing  architecture,  depicted  in  Figure  4,  is  reduced  to  the  Sea  Base-to-Objective 
system  depicted  on  the  right-hand  side  of  the  same  figure.  Two  vertical  connectors  (CH- 
53  and  V-22),  one  conceptual  surface  connector  (T-CRAFT),  and  one  ground  vehicle 
(MTVR)  have  been  selected  to  be  incorporated  into  this  proof  of  concept  involving 
sustainment  operations  from  the  Sea  Base.  The  T-CRAFT  is  a  fully  amphibious  vessel 
that  enables  rapid,  high  capacity  Sea  Base-to-Shore  transfer  of  materiel  and  personnel, 
and  is  also  self-deployable  from  an  advance  base  [4],  The  chosen  vehicles  operate  on 
each  of  the  three  legs  shown  in  the  Sea  Base-to-Objective  system  in  Figure  4  and  will 
illustrate  capability  tradeoffs  involving  a  mix  of  legacy  and  conceptual  vehicles.  Each 
vehicle  in  the  scenario  provides  troop  and  supply  sustainment  along  its  respective  mission 
segment. 
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Figure  4:  Sea  Base-to-Objective  system  architecture  based  on  [15] 

Identify  Capabilities  of  Interest 

The  capabilities  identified  in  Table  1  are  categorized  into  four  sets  of  design  parameters. 
The  Combat  Logistics  Force  (CLF),  which  delivers  troops  and  supplies  between  an 
advance  base  and  the  Sea  Base,  is  outside  the  scope  of  the  identified  system  and  is  thus 
represented  by  nominal  troop  and  supply  arrival  rates.  Troops  and  supplies  were 
discretized  into  JMICs1  (1,000  lb.  units)  for  the  purpose  of  modeling  based  on  weight 
capacity.  This  portion  of  the  Seabasing  concept  is  assumed  to  always  perform  correctly. 
The  physical  architecture  of  the  Sea  Base-to-Objective  system  is  explored  by  varying  the 
number  of  connectors  on  each  mission  segment.  A  T-CRAFT-like  capability  is 
incorporated  into  the  study  and  is  represented  by  dimensional  parameters  (DPs),  i.e., 
physical  properties  of  the  T-CRAFT  whose  values  determine  system  behavior  and 
structure  even  when  at  rest  [14].  Finally,  mission-specific  aspects  of  the  Sea  Base-to- 
Objective  system  are  explored  by  varying  the  sea  state,  mission  segment  distances,  and 
mission  duration.  The  ranges  for  each  of  the  parameters  in  Table  1  were  derived  from 
recent  Seabasing  literature,  e.g.,  [2],  [3],  [4],  [15],  [16],  [17],  [18],  [19],  [20], 


1  Joint  Modular  Intermodal  Container 
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Table  1:  Design  parameters  and  ranges 


Category 

Dimensional 

Parameter 

Design  Space 
Range 

Units 

CLF  Sustainment  to 
Sea  Base 

Food  Supply 

200-400 

JMICs/arrival 

Water  Supply 

4000  -  8000 

JMICs/arrival 

POL  Supply 

15000-20000 

JMICs/arrival 

Ammunition  Supply 

600  -  3000 

JMICs/arrival 

Other  Supply 

1500-2000 

JMICs/arrival 

Troops  Arriving 

35-100 

battalions  /arrival 

Arrival  Interval 

OO 

1 

^T 

<N 

hours 

Sea  Base  Physical 
Architecture 

No.  V-22 

24-48 

- 

No.  CH-53 

10-20 

No.  T-CRAFT 

4-10 

- 

No.  MTVR 

120  -  240 

T-CRAFT  Speed 

30-60 

knots  (kts) 

T-CRAFT  Capacity 

600-  1400 

JMICs 

Mission  Parameters 

Sea  State 

1 

o 

NATO  STANAG 
4194  Sea  State 

Distance  To  Shore 

100-250 

nautical  miles  (nm) 

Distance  To  Objective 

tn 

r--* 

1 

<N 

nm  ; 

Mission  Duration 

15-30 

days 

Identify  System  Metrics 


The  metrics  were  derived  from  literature  to  be  relevant  to  the  system  being  modeled  [21] 
and  are  in  the  form  of  MOPs,  i.e.,  measures  of  system  behavior  that  are  a  consequence  of 
specific  configurations  of  physical  elements  [14].  They  were  separated  into  two 
categories:  performance  metrics  and  economics  metrics.  The  metrics  are  summarized  in 
Table  2  and  discussed  below. 

For  successful  sustainment  operations  from  the  Sea  Base,  certain  sustainment  rates  of 
supplies  and  troops  must  be  met.  A  natural  measure  of  performance  for  sustainment  is 
the  mission-averaged  sustainment  rate,  defined  in  Equation  (1). 

„  .  ^  Total  Supplies  or  Troops  Delivered  ... 

Sustainment  Rate  = - — - - -  (1) 

Mission  Duration 

Connector  and  ground  vehicle  utilization  is  important  in  meeting  readiness  requirements; 
a  system  with  a  higher  sortie  generation  rate  exhibits  a  higher  utilization  and  hence  is 
more  capable  of  meeting  readiness  requirements  [22],  This  MOP  was  considered  in 
terms  of  the  sorties  per  vehicle  per  day  as  shown  in  (2). 


2  1,000  troops 
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SVD 


Sorties 

No.  Vehicles  x  Mission  Duration 


(2) 


Cost  was  considered  in  terms  of  fleet  acquisition  cost  and  operational  cost.  Fleet 
acquisition  cost  is  based  on  the  number  of  each  type  of  vehicle  and  the  reported 
acquisition  cost  per  connector  for  existing  connectors  or  the  estimated  prototype 
demonstrator  cost  for  conceptual  vehicles,  i.e.  T-CRAFT  [4],  [23],  [24],  [25].  The 
operational  cost  is  defined  as  mission  fuel  consumption  in  metric  tons  (MT)  normalized 
by  the  number  JMICs  delivered  during  the  mission  in  order  to  derive  the  fuel-equivalent 
cost  of  delivering  one  JMIC.  These  metrics  are  defined  in  (3)  and  (4),  respectively. 

Acquisition  Cost:  Acq  $  =  No.  Connectors  x  Unit  Cost  (3) 


„  . ^  *  Mission  Fuel  Consumption 

Operating  Cost:  Oper  $  = - - - 

JMICs  Delivered 


(4) 


Table  2:  System  Metrics 


Category 

Metric 

Nomenclature 

Units 

Performance 

Sea  Base 

Sustainment  to 
Shore 

Food  Sustainment 

Food  Sust. 

JMICs/day 

Water  Sustainment 

Water  Sust. 

JMICs/day 

POL  Sustainment 

POL  Sust. 

JMICs/day 

Ammunition  Sustainment 

Ammo  Sust. 

JMICs/day 

Other  Sustainment 

Other  Sust. 

JMICs/day 

Troops  Sustainment 

Troop  Sust. 

battalions/arrival 

Sortie 

Generation 

Rates 

V-22  Sorties  per  vehicle  per 
day 

V-22  SVD 

Sorties/No.  V- 

22/day 

CH-53  Sorties  per  vehicle 
per  day 

CH-53  SVD 

Sorties/No.  CH- 

53/day 

T-CRAFT  Sorties  per 
vehicle  per  day 

T-CRAFT  SVD 

Sorties/No.  T- 

CRAFT/day 

MTVR  Sorties  per  vehicle 
per  day 

MTVR  SVD 

Sorties/No. 

MTVR/day 

Economics 

Operational 

Cost 

V-22  operational  cost 

V-22  Oper  $ 

MT  fuel/JMIC 

CH-53  operational  cost 

CH-53  Oper  $ 

MT  fuel/JMIC 

T-CRAFT  operational  cost 

T-CRAFT  Oper  $ 

MT  fuel/JMIC 

MTVR  operational  cost 

MTVR  Oper  $ 

MT  fuel/JMIC 

Acquisition 

Cost 

Fleet  acquisition  cost 

Fleet  Acq  $ 

FY08  $M 
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4.2  STEP  2  Modeling  and  Simulation 

Identify  Systems  Functionality 

Standard  Department  of  Defense  Architecture  Framework  (DoDAF)  products  were  used 
to  guide  the  modeling  process  by  creating  representative  functional  networks  of  the  Sea 
Base-to-Objective  system  [26].  The  DoD  defines  an  architecture  as  “the  structure  of 
components,  their  relationship,  and  the  principles  and  guidelines  governing  their  design 
and  evolution  over  time.  ”  It  is  evident  that,  given  enough  resources,  all  possible 
architectures  could  be  examined  over  all  functions,  operational  activities,  and 
capabilities;  for  limited  resources,  the  architectures  determine  the  modeling  fidelity  that 
can  be  implemented  based  on  the  available  resources  and  desired  modeling  detail  [27], 
The  following  DoDAF  products  based  on  the  Universal  Naval  Task  List  [29]  were 
modified  from  previous  work  [28]: 


•  AV-1:  Overview  and  Summary  Information 

•  OV-1:  High  Level  Operational  Concept  Graphic 

•  OV-5:  Operational  Activity  Model 

•  SV-4a:  Systems  Functionality  Description 

•  SV-5a:  Operational  Activity  to  Systems  Functionality  Traceability  Matrix 


Figure  5  shows  a  graphical  depiction  of  the  Operational  Activity  to  Systems  Functionality 
Traceability  Matrix  (SV-5a),  which  maps  the  Operational  Activity  Model  (OV-5)  to  the 
Systems  Functionality  Description  (SV-4a).  This  mapping  identifies  the  transformation 
of  an  operational  need  into  a  purposeful  action  performed  by  the  system.  SV-5a  outlines 
the  functions  that  need  to  be  modeled  in  order  to  evaluate  the  capabilities  that  were 
identified  in  Table  1.  The  additional  DoDAF  products  are  located  in  “Annex  A  - 
DoDAF  Products.”  Fueling/re-fueling  is  not  present  in  the  architectures  due  to  the 
modeling  decision  to  implement  that  part  of  the  model  as  a  stand-alone  code. 


Figure  5:  Operational  Activity  to  Systems  Functionality  Traceability  Matrix  (SV-5a) 


Model  Development 

A  discrete  event  (DE)  model  was  chosen  to  represent  the  system  due  its  ability  to  model 
nonlinear,  dynamic  behavior  and  to  represent  hierarchies.  The  model  was  implemented 
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and  simulated  in  Extend®  6  [30],  a  modeling  and  simulation  environment  that  provides 
the  capability  to  directly  implement  architectures  as  models. 

Verification 

The  Extend  model  was  verified  against  the  DoDAF  products  to  ensure  that  the  intended 
functionality  was  implemented.  Boundary  cases  were  used  to  test  the  model  and  inputs 
were  traced  through  execution  to  ensure  that  there  were  no  bugs  in  the  logic  or  the 
underlying  code. 

Validation 

In  practice,  the  validation  process  is  difficult  for  simulations  of  future  concepts  where 
there  is  no  empirical  evidence  to  validate  against.  This  is  a  typical  situation  and  the  best 
practice  for  validation  of  this  type  of  simulation  is  to  independently  assess  each  of  the 
physics-based  components  to  ensure  that  the  physics  are  being  modeled  correctly.  For 
logic-based  components,  the  blocks  must  be  tested  under  various  conditions  to  ensure 
that  they  follow  the  correct  execution  paths.  It  is  then  usually  inferred  that  the  aggregated 
behavior  is  as  correct  as  possible  [27], 

The  only  physics-based  components  of  the  model  were  the  resistance  and  fuel  fraction 
codes,  which  were  used  to  predict  fuel  consumption  and  hence  operational  cost.  The  T- 
CRAFT  was  modeled  as  a  surface  effect  ship;  resistance  was  calculated  using  standard 
air-cushion  vehicle  powering  calculations  [31].  The  resistance  code  was  compared  to 
experimental  data  obtained  from  scale  model  tests  of  the  SES  100B  [32];  the  comparison 
is  shown  in  Figure  6.  The  discrepancy  in  the  resistance  at  higher  Froude  numbers  is  due 
to  the  assumption  that  the  draft  of  the  surface  effect  ship  is  constant  with  speed;  in  reality, 
the  draft  decreases  with  speed  thus  resulting  in  a  decrease  in  resistance,  as  seen  in  the 
experimental  data.  The  Breguet  range  equation  was  used  to  predict  the  empty  weight  fuel 
fraction  for  propeller-driven  aircraft  and  rotorcraft  [33].  The  model  was  compared  to 
existing  vehicles  and  the  results  are  presented  in  Figure  7.  All  logic-based  components 
were  found  to  perform  correctly;  validation  results  for  the  model  logic  are  not  presented 
here. 


Figure  6:  Comparison  of  resistance  code  with 
experimental  data  from  [32] 
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Figure  7:  Aircraft  and  rotorcraft  empty 
weight  fuel  fraction  prediction 


4.3  STEP  3  Development  of  Tradeoff  Environment 

Surrogate  Model  Development 

To  address  the  long  run-times  associated  with  DE  simulations,  surrogate  models  were 
constructed  to  enable  rapid  trade  studies.  The  DE  model  was  executed  for  up  to  30  days 
of  simulation  time,  which  took  on  the  order  of  one  minute  on  a  current  desktop  computer. 
The  development  of  the  surrogate  models  began  with  the  creation  of  a  DoE  table.  Due  to 
the  combination  of  continuous  and  discrete  inputs,  a  custom  DoE  was  created  consisting 
of  the  following  designs:  (1)  256  full  factorial  type  cases  at  3  levels  to  capture  the  main 
effects  of  the  model;  (2)  32  Latin  Hypercube  cases  to  sample  the  interior  design  space  of 
the  model;  (3)  32  random  cases  to  evaluate  the  model  representation  error.  The  DoE 
table  was  constructed  using  SAS  JMP®  7.1  [34],  an  interactive  statistical  analysis 
environment.  Artificial  Neural  Networks  (ANNs)  were  selected  as  the  surrogate  models 
in  order  to  capture  any  potentially  highly  nonlinear  and/or  discrete  behavior  that  would 
arise  in  the  responses.  Using  Basic  Regression  Analysis  of  Integrated  Neural  Networks 
(BRAINN)  2.1  for  MATLAB  [35],  ANNs  were  trained  and  their  functional  forms  were 
passed  into  JMP. 

BRAINN  provides  the  means  to  validate  the  surrogate  models  against  the  original  model 
data  via  five  “goodness  of  fit”  tests:  R2,  actual  by  predicted  plot,  residual  by  predicted 
plot,  model  fit  error,  and  model  representation  error.  These  tests  are  discussed  in  detail  in 
[10]  and  [35].  In  general,  the  surrogate  models  should  predict  the  actual  data  points  as 
closely  as  possible,  residuals  should  be  as  small  as  possible  and  randomly  distributed,  and 
errors  should  approximate  a  standard  normal  distribution.  Figure  8  shows  the  validation 
results  for  the  ANN  representing  troop  sustainment;  additional  results  are  presented  in 
Annex  B. 
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Actual  by  Predicted  Residual  by  Predicted 


Figure  8:  ANN  validation  tests  for  troop  sustainment 

With  the  goodness  of  fit  tests  accepted,  the  next  step  is  to  visualize  the  design  space. 
JMP  provides  a  suite  of  visualization  tools  that  enable  visual  tradeoffs  and  probabilistic 
analyses  to  be  performed.  Figure  9  depicts  the  final  modeling  and  simulation  process 
employed  for  this  study. 


START 


Paramefric  Tradeoff  trivironment 


^  |  *  «-**  +  -  z  | 

Discrete  Event  Model 
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leural  Network 
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*JMP  :  Statistical  analysis  tool 


Figure  9:  Modeling  and  simulation  process 
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Visualization 

The  first  tool  used  to  visualize  the  tradeoff  environment  was  the  prediction  profiler.  JMP 
was  used  to  produce  an  interactive  prediction  profiler  as  shown  in  Figure  10.  Figure  10 
only  shows  the  portion  of  the  profiler  that  relates  to  the  T-CRAFT;  the  full  prediction 
profiler  is  shown  in  Annex  C.  A  prediction  profiler  is  a  matrix  of  bivariate  plots  that 
enables  the  designer  to  check  the  behavior  of  the  model  on  a  one-to-one  basis  with 
respect  to  the  design  parameters  and  MOPs,  and  to  establish  the  sensitivities  of  each 
metric  to  those  design  parameters.  Thus,  the  designer  can  validate  the  model  from  a 
physical  standpoint  and  determine  the  main  drivers  for  each  measure  of  performance. 
For  example,  for  the  settings  shown  in  Figure  10,  the  T-CRAFT  operational  cost 
increases  with  increasing  distance  to  shore,  which  is  correct  from  a  physical  standpoint 
based  on  (4).  One  can  also  note  that  troop  sustainment  is  most  sensitive  to  the  troop 
arrival  rate  at  the  Sea  Base.  Figure  10  is  a  screenshot  of  the  prediction  profiler;  in  the 
actual  program,  the  designer  can  move  the  crosshairs  and  instantaneously  assess  the 
impacts  of  new  designs  and  capabilities.  For  this  reason,  the  prediction  profiler  is  useful 
in  performing  parametric  design  trades  and  optimization  via  desirability  functions,  e.g, 
[36],  [37], 
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Figure  10:  T-CRAFT  performance  prediction  profiler 


4.4  STEP  4  Design  Space  Exploration 

The  first  step  in  the  design  space  exploration  is  to  develop  requirements  on  the  MOPs  for 
the  mission  being  studied.  The  required  MOPs  are  derived  from  the  top-level  mission 
goals,  in  this  case,  the  five  top-level  Seabasing  MOPs.  The  Sea  Base  must  be  able  to 
sustain  up  to  two  MEBs  for  an  indefinite  amount  of  time.  For  this  application,  MEB 
sustainment  is  treated  as  troops  and  POL  per  day.  Notional  MEB  sustainment 
requirements  of  1200  JMICs  of  POL  per  day  and  10  battalions  of  troops  per  day  will  be 
used. 
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Capability  tradeoffs  are  visualized  using  a  scatterplot  matrix,  which  shows  correlations 
between  selected  MOPs  via  the  red  correlation  ellipses.  The  scatterplot  matrix  for  this 
application  is  shown  in  Figure  11.  Only  the  MOPs  relating  to  the  T-CRAFT  and  the  POL 
and  troop  sustainment  are  shown;  the  full  scatterplot  matrix  can  be  found  in  Annex  C  - 
Unabridged  Tradeoff  Environment.  The  design  points  shown  are  the  actual  data  points 
from  the  DoE;  however,  the  surrogate  models  may  be  used  to  rapidly  generate  additional 
design  points. 


Troop  Sust.  POLSust. 

(battalions/day)  (JMICS/day)  T-CRAFT  SVD 


Figure  11:  Scatterplot  matrix 


The  next  step  is  to  impose  the  requirements  on  the  design  space.  Once  the  requirements 
are  imposed  (Figure  12),  the  designs  may  be  filtered  to  hide  infeasible  designs. 
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Figure  12:  Scatterplot  matrix  with  required  MOPs  superposed 

This  process  of  down  selection  results  in  Figure  13.  Figure  13  shows  that  the  majority  of 
the  designs  do  not  meet  the  required  MOPs.  This  is  indicative  that  one  or  both  of  the 
requirements  are  constraining  the  design  space. 
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Figure  13:  Down-selected  scatterplot  matrix 

The  utility  of  this  approach  becomes  apparent  when  it  is  coupled  with  a  Monte  Carlo 
simulation  (MCS)  of  the  system.  The  purpose  of  MCS  in  this  application  is  to  quantify 
the  probability  of  success,  or  confidence,  of  meeting  the  required  MOPs  with  designs  that 
are  within  the  ranges  of  the  design  parameters.  The  design  parameters  are  treated  as 
uniform  random  variables  to  enable  an  unbiased  exploration  of  the  design  space.  In 
practice,  MCS  requires  approximately  10,000  runs  per  input  variable  to  produce 
dependable  results.  The  MOPs  are  visualized  using  cumulative  distribution  functions 
(CDFs).  A  low  confidence  in  meeting  a  required  MOP  implies  that  that  requirement  is 
constraining  the  system.  This  is  indicative  that  one  or  more  changes  need  to  be 
implemented  into  the  system  in  order  to  meet  the  required  MOPs  with  a  higher 
confidence.  The  system  architectures  and  model  must  be  revisited  to  determine  the  cause 
of  failure.  For  the  example  presented  here,  it  can  be  shown  that  the  cause  of  failure  in 
confidently  meeting  the  troop  sustainment  requirement  is  due  to  the  modeling  assumption 
that  only  one  connector  of  each  type  is  loaded  at  a  time.  In  reality,  multiple  connectors 
are  loaded  simultaneously;  loading  in  queue  is  not  a  realistic  representation  of 
amphibious  operations.  Implementing  this  change  in  the  system  model  and  repeating 
Steps  3  and  4  may  shift  the  distribution  in  Figure  15  to  the  right,  resulting  in  a  higher 
percentage  of  feasible  designs.  The  designer  has  the  added  benefit  of  determining  if  such 
a  change  is  statistically  significant  to  the  performance  of  the  system  through  the  use  of 
this  probabilistic  approach. 
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Figure  14:  Probability  of  meeting  required  Figure  15:  Probability  of  meeting  required 

POL  sustainment  troop  sustainment 

5  Conclusions 

A  parametric  and  capability-based  tradeoff  environment  is  developed  that  enables  rapid, 
probabilistic  design  space  exploration  via  surrogate  models.  This  quantitative  framework 
can  be  used  to  assess  the  impacts  of  new  technologies  and  systems  on  the  system-of- 
systems.  The  proposed  approach  has  been  applied  to  the  Sea  Base-to-Objective  system  to 
determine  the  performance  and  economics  impacts  of  a  conceptual  connector,  i.e.,  T- 
CRAFT,  as  well  as  architectural  changes  on  the  system.  The  process  enables  designers  to 
utilize  a  “design  for  capability”  approach  and  to  translate  theater-level  goals  into  specific 
asset  and  architectural  requirements  to  accomplish  those  goals.  The  design  space  for  the 
system  is  explored  probabilistically  to  determine  the  feasibility  in  meeting  required 
performance  and  economics  metrics.  This  assessment  indicates  to  designers  which 
requirements  are  constraining  the  design  and  provides  guidance  to  focus  resources  for  the 
next  design  iteration.  Compared  with  previous  methods,  the  proposed  approach  enables 
decision  makers  to  make  informed  decisions  during  the  requirements  definition  and 
conceptual  phases.  It  also  offers  the  potential  to  reduce  the  time  and  cost  needed  to 
develop  a  design  that  meets  or  exceeds  customer  requirements. 

6  Recommendations 

The  proposed  approach  can  be  applied  to  different  levels  of  modeling  to  include 
subsystems,  systems,  and  systems-of-systems.  The  surrogate  models  that  are  developed 
for  each  level  may  be  employed  in  higher  level  models  for  rapid  capability-based 
tradeoffs.  Using  decomposition  of  the  SoS  and  aggregation  of  the  surrogate  models,  a 
high-fidelity  Seabasing  model  may  be  developed  to  perform  inverse  design  in  order  to 
identify  what  technologies,  platforms,  and  MOPs  are  needed  to  meet  the  high-level 
MOEs.  For  instance,  designers  can  perform  a  top-down  decomposition  of  the  goal 
‘‘Provide  National  Security”  to  obtain  the  physical  systems  that  are  required  to 
accomplish  this  goal.  The  proposed  approach  provides  the  level  of  abstraction  and 
transparency  that  is  necessary  to  enable  decision  makers  to  translate  a  Navy  vision  into 
engineering  requirements  via  a  unified  tradeoff  environment  as  illustrated  in  Figure  16. 
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Figure  16:  Unified  tradeoff  environment 
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Annex  A  -  DoDAF  Products 
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Figure  17:  AV-1  Overview  and  Summary  Information 
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Figure  18:  OV-1  High  Level  Operational  Concept  Graphic 
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Figure  19:  OV-5  Operational  Activity  Model 


Figure  20:  SV-4a  Systems  Functionality  Description 


Figure  21:  SV-5a  Operational  Activity  to  Systems  Functionality  Traceability  Matrix 
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Annex  B  -  Neural  Network  Validation  Results 


Figure  25:  Ammunition  sustainment 


Figure  23:  Water  sustainment 


Figure  26:  POL  sustainment 


Figure  24:  Food  sustainment 


Figure  27:  Other  sustainment 
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Figure  28:  V-22  SVD 


Figure  31:  MTVR  SVD 


Figure  29:  CH-53  SVD 


Figure  32:  V-22  operational  cost 
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Figure  30:  T-CRAFT  SVD 


Figure  33:  CH-53  operational  cost 
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Figure  34:  T-CRAFT  operational  cost 


Figure  35:  MTVR  operational  cost 


27 


(FY08  $M) 


Naval  Surface  Warfare  Center  Carderock  Division 
Naval  Research  Enterprise  Intern  Program 
A  Quantitative  Framework  To  Assess  The  Impacts  Of  New  Technologies  And  Systems  On  The 

Seabasing  Concept 


Annex  C  -  Unabridged  Tradeoff  Environment 


Figure  36:  Scatterplot  matrix 
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Figure  37:  Prediction  profile 
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